Measurement Invariance in Intellectual and Developmental Disability Research.

Measurement invariance (MI) is a psychometric property of an instrument indicating the degree to which scores from an instrument are comparable across groups. In recent years, there has been a marked uptick in publications using MI in intellectual and developmental disability (IDD) samples. Our goal here is to provide an overview of why MI is important to IDD researchers and to describe some challenges to evaluating it, with an eye towards nudging our subfield into a more thoughtful and measured interpretation of studies using MI.

Refining our Understanding of Anxiety in Autistic Youth: Examining the Role of Behavioral Inflexibility.

Prior research has demonstrated that cognitive inflexibility is associated with anxiety in autistic individuals. Everyday patterns of behavioral inflexibility (e.g. observable inflexible behavior in the context of the need to change or adapt and that is manifested in real-world everyday settings) is common in autism and can be distinguished from performance on discrete cognitive tasks that tap flexible attention, learning, or decision-making. The purpose of this study was to extend this prior work on inflexibility in autism but with measures specifically developed with input from stakeholders (caregivers and clinicians) for autistic youth designed to measure everyday behavioral inflexibility (BI). We characterized anxiety in a large sample of autistic (N = 145) and non-autistic youth (N = 91), ages 3 to 17 years, using the Parent Rated Anxiety Scale for Autism Spectrum Disorder (PRAS-ASD). Further, we sought to understand how BI, measured via the Behavioral Inflexibility Scale (BIS), predicted anxiety compared to other variables known to increase anxiety in youth (chronological age, IQ, autism diagnosis, assigned sex at birth). Autistic youth had higher parent-related anxiety and BI compared to non-autistic youth. BI was the strongest predictor of anxiety scores, irrespective of diagnosis. Overall, our findings highlight the importance of BI to the understanding of anxiety in autistic youth.

Development and Preliminary Validation of the Accommodations & Impact Scale for Developmental Disabilities.

The lives of caregivers can be deeply impacted by having a child with a developmental disability (DD). To offset those impacts, caregivers may engage in accommodations, or strategies to bolster everyday functioning. The nature and extent of these accommodations can provide insight into how the family is doing and what supports are needed from a family-centered perspective. This paper presents the development and preliminary validation of the Accommodations & Impact Scale for Developmental Disabilities (AISDD). The AISDD is a rating scale that measures day-to-day accommodations and impacts of raising a child with a DD. A sample of 407 caregivers of youth with DDs (Mage = 11.7 years; 63% males) completed the AISDD, along with measures of caregiver strain, daily challenges, child adaptive behavior, and behavior and emotional regulation. The AISDD is a unidimensional, 19-item scale with excellent internal consistency (ordinal alpha = .93) and test-retest (ICC = .95) reliability. Scores were normally distributed and sensitive to age (r = - .19), diagnosis (ASD + ID > ASD > ID), adaptive functioning (r = - .35), and challenging behaviors (r = .57). Finally, the AISDD showed excellent convergent validity with similar measures of accommodations and impacts. These findings support the use of the AISDD as a valid and reliable tool for measuring accommodations among caregivers of individuals with DDs. This measure shows promise in its ability to identify which families may need additional support for their children.

Characterizing Sleep Problems in 16p11.2 Deletion and Duplication.

Studies of 16p11.2 copy number variants (CNVs) provide an avenue to identify mechanisms of impairment and develop targeted treatments for individuals with neurodevelopmental disorders. 16p11.2 deletion and duplication phenotypes are currently being ascertained; however, sleep disturbances are minimally described. In this study, we examine sleep disturbance in a well-characterized national sample of 16p11.2 CNVs, the Simons Foundation Autism Research Initiative (SFARI) database of youth and adults (n = 692). Factor analyses and multilevel models of derived sleep questionnaires for youth (n = 345) and adults (n = 347) indicate that 16p11.2 carriers show elevated sleep disturbance relative to community controls. Non-carrier family members also show elevated sleep disturbance. However, sleep duration does not differ between carriers and controls. Further studies of sleep in 16p11.2 are needed.

Examining Sensitivity to Developmental Changes on the Behavioral Inflexibility Scale.

PURPOSE: The study objective was to determine if the validated Behavioral Inflexibility Scale (BIS) is sensitive to the detection of developmental changes in inflexibility in a sample of autistic children. METHODS: Parents of autistic children (n = 146, 3-17 years) completed the BIS at two time points, one year apart, to examine change. RESULTS: The findings indicate the BIS is sensitive to the detection of developmental changes and that child-level variables are not associated with those changes. Children's Time 1 BIS scores predicted children's severity on an independent outcome measure. Finally, a relationship between total services children were receiving and change in BIS scores over time was not found. CONCLUSION: The findings suggest the BIS is a reasonable candidate for consideration as an outcome measure.

Putting "ME" into measurement: Adapting self-report health measures for use with individuals with intellectual disability.

BACKGROUND: Self-report is important for measuring health outcomes; however, most research in intellectual disability (ID) relies on proxy report. The lack of cognitively accessible measures is one barrier to accurate self-reporting by individuals with ID. AIMS: This paper describes the process of adapting self-report measures of health status, health-related quality of life, and environment for use by individuals with ID and presents evidence on their usability (accessibility), usefulness (independent self-report), and reliability (internal consistency and test-retest). METHODS AND PROCEDURES: We used an inclusive research approach, in which we collaborated with adults with ID to revise, cognitively test, and pilot test cognitively accessible self-report measures. Technology supported the independent completion of measures. We assessed usability, usefulness, and reliability of these measures in 41 adults with ID. OUTCOMES AND RESULTS: The resulting measures are useful (independently completed) and usable (elicit a range of responses), with modest reliability (internal consistency and test-retest). CONCLUSIONS AND IMPLICATIONS: Self- report by adults with ID is feasible. A key element of this measure adaptation process was engaging adults with ID. More research is needed to understand the reliability and validity of the adapted measures and the characteristics of the population for whom they are most usable.

Brief report: replication of the psychometric characteristics of the behavioral inflexibility scale in an independent sample.

The Behavioral Inflexibility Scale (BIS) is a recently developed measure of behavioral inflexibility, defined as rigid patterns of behavior that contrast with the need to be flexible when the situation calls for it. In this study, we sought to replicate previous findings on the psychometric properties of the BIS in a community sample. Data for this study were collected using in-person assessments of 163 autistic and 95 non-autistic children ages 3-17 and included the BIS, measures of social-communication ability and repetitive behaviors, and an assessment of cognitive ability. Our findings replicate the psychometric properties of the BIS, indicating that the measure is a valid measure of behavioral inflexibility in ASD.

Measuring the Functional Impact of Behavioral Inflexibility in Children with Autism Using the Behavioral Inflexibility Scale: Clinical Interview (BIS-CI).

For individuals with autism spectrum disorder (ASD), behavioral inflexibility can affect multiple domains of functioning and family life. The objective of this study was to develop and validate a clinical interview version of the Behavioral Inflexibility Scale. Trained interviewers conducted interviews with parents of 144 children with ASD and 70 typically developing children (ages: 3-17 years). Using exploratory factor analysis, the Behavioral Inflexibility Scale-Clinical Interview (BIS-CI) was found to be unidimensional. Reliability data indicated the measure was internally consistent (α = 0.80), achieved excellent inter-rater reliability (ICC = 0.97) and test-retest reliability (ICC = 0.87). These findings demonstrate that the BIS-CI is a reliable and valid measure to determine the functional impact of behavioral inflexibility.

Evaluating the Feasibility of The NIH Toolbox Cognition Battery for Autistic Children and Adolescents.

This study evaluates the feasibility of the NIH Toolbox Cognition Battery (NIH-TCB) for use in autism spectrum disorder (ASD). 116 autistic children and adolescents and 80 typically developing (TD) controls, ages 3-17 years, completed four NIH-TCB tasks related to inhibitory control, cognitive flexibility, processing speed, and episodic memory. While the majority of autistic and TD children completed all four tasks, autistic children experienced greater difficulties with task completion. Across autistic and TD children, performance on NIH-TCB tasks was highly dependent on IQ, but significant performance differences related to ASD diagnosis were found for two of four tasks. These findings highlight the potential strengths and limitations of the NIH-TCB for use with autistic children.

Evaluating the Use of Self-reported Measures in Autistic Individuals in the Context of Psychiatric Assessment: A Systematic Review.

The current review examined the use of self-report measures in autistic individuals in the context of psychiatric assessments. It focused on inter-rater agreement, internal consistency, test-retest reliability, and criterion validity with clinical diagnoses. It also gathered information on constructs measured, the nature of the samples, and the quality of the studies. Thirty-six out of 10,557 studies met inclusion criteria. We found that the majority of studies (1) targeted young people with average or above average cognitive abilities, (2) measured anxiety symptoms, and (3) evaluated parent-child agreement. More studies are needed on individuals with lower cognitive abilities, adults, and other constructs. Studies assessing criterion validity and test-retest reliability are also needed.

Direct observation in a large-scale randomized trial of parent training in children with autism spectrum disorder and disruptive behavior.

A direct observation strategy (Standardized Observation Analogue Procedure, SOAP) was used in a large-scale randomized trial of parent training versus parent education in young children with autism spectrum disorder (ASD) and disruptive behavior. The 16-minute SOAP, modified from an earlier version of this same measure, included parentchild interaction to assess child behavior in a clinical laboratory setting. Despite study entry criteria for all child participants requiring moderate levels of disruptive behavior in this project, 126 of 168 children with complete SOAP data at baseline showed no disruptive behavior on this measure. Although the primary purpose of the study was to determine whether the SOAP could detect differences between the two conditions (i.e.,parent training (PT) and parent education (PE)), baseline observation data was not consistent with parent ratings at baseline or subsequent follow up visits, leaving little room to demonstrate improvement with this observation measure. This and the challenging, time-consuming and resource intensive effort involved in using such a measure in a large randomized scale trial, raises fundamental questions about the validity of the SOAP as an outcome measure in such a study. Further consideration related to the feasibility and practicality of using direct observation as a primary measure in larger scale efforts overall are also discussed.

Behavioral Inflexibility Across Two Neurogenetic Conditions: Down Syndrome and Fragile X Syndrome.

Behavioral inflexibility (BI) has been highlighted to occur across genetic and neurodevelopmental disorders. This study characterized BI in two common neurogenetic conditions: Fragile X syndrome (FXS) and Down syndrome (DS). Caregivers of children with FXS (N = 56; with ASD = 28; FXS only = 28) and DS (N = 146) completed the Behavioral Inflexibility Scale (BIS) via an online survey. Total BIS scores were higher in FXS+ASD than both FXS only and DS (p <.001). Most endorsed items were similar across the three groups, but scores were higher in the FXS+ASD group. In all groups, BI associated with other clinical variables (receptive behaviors, anxiety, social communication). The current data suggest that BI is variable across neurogenetic conditions and higher in individuals with comorbid ASD.

Using Parent Target Problem Narratives to Evaluate Outcomes in Children with Autism Spectrum Disorder.

This paper examines the reliability and validity of parent target problems (PTPs) in a multi-site randomized controlled trial of parent training (PT) versus psychoeducation (PEP) in children (150 boys, 19 girls; mean age 4.7 ± 1.2 years) with autism spectrum disorder (ASD) and disruptive behavior. At baseline, treatment blind, independent evaluators asked parents to nominate the child's top two problems. Each problem was documented in a brief narrative. Narratives were reviewed and revised at follow-up visits during the six-month trial. When the trial was completed, five judges, blind to treatment condition, independently rated change from baseline on a 9-point scale (1 = normal; 2 = markedly improved; 3 = definitely improved; 4 = equivocally improved; 5 = no change; 6 = possibly worse; 7 = definitely worse; 8 = markedly worse; 9 = disastrously worse) at Weeks 8, 12, 16, and 24 (inter-rater intraclass correlation = 0.78). PTP scores for the two target problems were averaged across the five raters, yielding a mean score for each child at each time point. Mean PTP scores showed improvement in both treatment groups over the 24-week study. Compared to PEP, PTP ratings showed a steeper decline in PT based on significant interaction of group and time (t(df) = 2.14(155.9), p = 0.034; Week 24 effect size = 0.75). In categorical analysis, we compared cutoffs mean PTP scores of 3.0 (definitely improved), 3.25, and 3.5 with the positive response rate on the Clinical Global Impressions-Improvement scale from the original study. Sensitivities ranged from 52-78%. PTP narratives offer a systematic, reliable, and valid way to track child-specific outcomes in clinical trials and clinical practice.

Predictors of Caregiver Strain for Parents of Children with Autism Spectrum Disorder.

Parents of children with autism spectrum disorder (ASD) face higher levels of caregiver strain compared to parents of children with other disabilities. This study examined child clinical features that predict high levels of caregiver strain for 374 parents of children with ASD. Caregiver strain was measured using the Caregiver Strain Questionnaire (CGSQ) objective, subjective internalized, and subjective externalized subscales. Confirmatory factor analysis indicated an acceptable fit for the original CGSQ three-factor solution. The strongest child predictors across CGSQ subscales were: disruptive behavior for objective strain, autism severity and disruptive behavior for subjective internalized strain, and oppositional behavior and hyperactivity for subjective externalized strain. Individualized interventions that attend to specific elements of parental strain may reduce strain and improve family wellbeing.

Reliability and validity of the Pediatric Anxiety Rating Scale modified for autism spectrum disorder.

LAY ABSTRACT: Many youth with autism spectrum disorder have anxiety, but it can be difficult to assess anxiety with existing measures. We modified the Pediatric Anxiety Rating Scale for youth with autism spectrum disorder and tested the new measure in a group of 116 youth (age: 5-17 years) with autism spectrum disorder. The Pediatric Anxiety Rating Scale for youth with autism spectrum disorder is an interview that a clinician usually completes with the child and parent together. We modified the interview questions and scoring instructions based on feedback from parents of children with autism spectrum disorder and from a panel of experts in autism spectrum disorder and anxiety. Unlike many other anxiety measures, the Pediatric Anxiety Rating Scale for youth with autism spectrum disorder relies less on a child's verbal expression of anxiety and more on signs that a parent can easily observe. Training clinicians to administer and score the Pediatric Anxiety Rating Scale for youth with autism spectrum disorder was uncomplicated, and raters showed excellent agreement on video-recorded interviews. Youth who were not currently in treatment for anxiety had stable Pediatric Anxiety Rating Scale for youth with autism spectrum disorder scores with repeat measurement over a 1-month period. The Pediatric Anxiety Rating Scale for youth with autism spectrum disorder is a useful clinician-rated measure of anxiety in youth with autism spectrum disorder and fills a gap for assessing anxiety in this population.

A Pilot Study of Cardiovascular Reactivity in Children With Autism Spectrum Disorder.

In preparation for a larger case-control study of children with autism spectrum disorder (ASD) and anxiety, we conducted a pilot study using a noninvasive electrocardiographic device to measure cardiovascular reactivity in 10 children (age range 9-14) with ASD. The 45-minute procedure included 6 conditions: baseline rest, an interview about school, interim rest, an unfair computerized ball-toss game followed by a fair version of the game, and a final rest. Data were successfully collected for 95% of all conditions. Omnibus Skillings-Mack tests suggested that heart rate variability variables including mean heart rate, mean RR interval, and root mean square of successive differences showed statistically significant variation across conditions. The procedure appears feasible and may be an informative biomarker of anxiety in ASD.

Development of the Behavioral Inflexibility Scale for Children with Autism Spectrum Disorder and Other Developmental Disabilities.

Behavior inflexibility (BI) refers to rigid patterns of behavior that contrast with the need to be adaptable to changing environmental demands. We developed a parent-reported outcome measure of BI for children with autism spectrum disorder (ASD) and other developmental disabilities with a multi-step iterative process. A pool of 62 candidate items was generated through expert panel feedback, review of existing scales and focus groups. A consensus process was used to generate the final 38 items. Parents of 943 children (age range, 3-18 years; average, 11.4 years; 79% boys) with ASD completed an online survey. One hundred thirty-three parents rated their child twice within 3 weeks (average = 16.5 days). A series of factor analyses suggested that the 38 items measured a single construct. Scores had a weak correlation with level of functioning (-0.12) and did not differ based on sex. Scores had a negligible correlation with age (-0.07), although measurement invariance was not supported. The mean total score for the Behavioral Inflexibility Scale (BIS) was normally distributed. Internal consistency was α = 0.97 and temporal stability was r = 0.92. Correlations with parent ratings on the subscales of the Repetitive Behavior Scale-Revised varied from 0.48 to 0.89. The correlation with parent ratings on the Social Communication Questionnaire total score was 0.52. Our data show that BI in children with ASD ranges significantly from mild to severe and that the 38-item BIS is valid and reliable. Autism Res 2020, 13: 489-499. © 2020 The Authors. Autism Research published by International Society for Autism Research published by Wiley Periodicals, Inc. LAY SUMMARY: We developed a parent-completed rating scale of behavior inflexibility (BI) for children with developmental disabilities using a multistep process. The Behavioral Inflexibility Scale (BIS) contains 38 questions rated on a 6-point scale. Parents of 943 children with autism spectrum disorder (ASD) completed an online survey. We examined associations between the BIS and other scales and demographic variables. The BIS is valid and reliable. BI in children with ASD ranges from mild to severe.

Anxiety in 3- to 7-year-old children with autism spectrum disorder seeking treatment for disruptive behavior.

Anxiety is a common and impairing problem in children with autism spectrum disorder, but little is known about it in preschool children with autism spectrum disorder. This article reports on the characteristics of anxiety symptoms in young children with autism spectrum disorder using a parent-completed rating scale. One hundred and eighty children (age 3-7 years) participated in a clinical trial of parent training for disruptive behaviors. Anxiety was measured as part of pre-treatment subject characterization with 16 items from the Early Childhood Inventory, a parent-completed scale on child psychiatric symptoms. Parents also completed other measures of behavioral problems. Sixty-seven percent of children were rated by their parents as having two or more clinically significant symptoms of anxiety. There were no differences in the Early Childhood Inventory anxiety severity scores of children with IQ < 70 and those with ⩾70. Higher levels of anxiety were associated with severity of oppositional defiant behavior and social disability. Anxiety symptoms are common in preschoolers with autism spectrum disorder. These findings are consistent with earlier work in school-age children with autism spectrum disorder. There were no differences in anxiety between children with IQ below 70 and those with IQ of 70 and above. Social withdrawal and oppositional behavior were associated with anxiety in young children with autism spectrum disorder.

Development of the Parent-Rated Anxiety Scale for Youth With Autism Spectrum Disorder.

OBJECTIVE: Anxiety is common in youth with autism spectrum disorder (ASD). There is no accepted outcome measure for anxiety in this population. METHOD: Following a series of focus groups with parents of youth with ASD, we generated 72 items (scored 0-3). Parents of 990 youth with ASD (aged 5-17 years; 80.8% male) completed an online survey. Factor analysis and item response theory analyses reduced the content to a single factor with 25 items. Youth with at least mild anxiety (n = 116; aged 5-17 years; 79.3% male) participated in a comprehensive clinical assessment to evaluate the validity and reliability of the 25-item Parent-Rated Anxiety Scale for ASD (PRAS-ASD). RESULTS: In the online sample, the mean PRAS-ASD score was 29.04 ± 14.9 (range, 0-75). The coefficient α was 0.93. The item response theory results indicated excellent reliability across a wide range of scores with low standard errors. In the clinical sample (n = 116), the PRAS-ASD mean was 31.0 ± 15.6 (range, 1-65). Pearson correlations with parent ratings of ASD symptom severity, repetitive behavior, and disruptive behavior ranged 0.33 to 0.66, supporting divergent validity of the PRAS-ASD. Pearson correlation with a parent-rated measure of anxiety used in the general pediatric population of 0.83 supported convergent validity. A total of 40 participants (32 boys, 8 girls; mean age, 11.9 ± 3.4 years) returned at time 2 (mean, 12.2 days) and time 3 (mean, 24.2 days). Intraclass correlation showed test-retest reliabilities of 0.88 and 0.86 at time 2 and time 3, respectively. CONCLUSION: The 25-item PRAS-ASD is a reliable and valid scale for measuring anxiety in youth with ASD.

An exploration of concomitant psychiatric disorders in children with autism spectrum disorder.

OBJECTIVE: We explored patterns of concomitant psychiatric disorders in a large sample of treatment-seeking children and adolescents with autism spectrum disorder (ASD). METHODS: Participants were 658 children with ASD (age 3-17 years; mean = 7.2 years) in one of six federally-funded multisite randomized clinical trials (RCT) between 1999 and 2014. All children were referred for hyperactivity or irritability. Study designs varied, but all used the Child and Adolescent Symptom Inventory or Early Childhood Inventory to assess Attention Deficit Hyperactivity Disorder (ADHD), Oppositional-Defiant Disorder (ODD), Conduct Disorder (CD), Anxiety Disorders, and Mood Disorders. In addition, several measures in common were used to assess demographic and clinical characteristics. RESULTS: Of the 658 children, 73% were Caucasian and 59% had an IQ >70. The rates of concomitant disorders across studies were: ADHD 81%, ODD 46%, CD 12%, any anxiety disorder 42%, and any mood disorder 8%. Two or more psychiatric disorders were identified in 66% of the sample. Of those who met criteria for ADHD, 50% also met criteria for ODD and 46% for any anxiety disorder. Associations between types of concomitant disorders and a number of demographic and clinical characteristics are presented. CONCLUSION: In this well-characterized sample of treatment-seeking children with ASD, rates of concomitant psychiatric disorders were high and the presence of two or more co-occurring disorders was common. Findings highlight the importance of improving diagnostic practice in ASD and understanding possible mechanisms of comorbidity.